Skip to content

[ROCm] Rework ROCm build to use ROCm version instead of HIP version - #1888

Open
sstamenk wants to merge 5 commits into
bitsandbytes-foundation:mainfrom
sstamenk:fix/rocm-build-rework
Open

[ROCm] Rework ROCm build to use ROCm version instead of HIP version#1888
sstamenk wants to merge 5 commits into
bitsandbytes-foundation:mainfrom
sstamenk:fix/rocm-build-rework

Conversation

@sstamenk

@sstamenk sstamenk commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Use the ROCm release version consistently when naming and selecting bitsandbytes native libraries. HIP and ROCm versions can diverge, so relying on hipconfig or torch.version.hip alone can select a missing or incompatible binary.

Changes

Build and packaging

  • Keep the existing HIP build backend (COMPUTE_BACKEND=hip and BUILD_HIP).
  • Detect the ROCm release from ${ROCM_PATH}/.info/version, ${ROCM_PATH}/core/.info/version, or rocm-sdk version.
  • Fail with guidance when the ROCm version cannot be detected; -DROCM_VERSION=<version> remains an optional override and accepts dotted or compact forms.
  • Require ROCM_PATH for Windows wheel-based SDK builds; Linux warns before defaulting to /opt/rocm.
  • Pass the CI matrix ROCm version explicitly to CMake and verify that the expected versioned artifact was produced.
  • Require hipBLASLt for ROCm builds and remove the obsolete NO_HIPBLASLT path.

Runtime library selection

  • Prefer torch.version.rocm; warn and fall back to torch.version.hip when the ROCm field is unavailable.
  • Accept dotted or compact values for both BNB_ROCM_VERSION and BNB_CUDA_VERSION.
  • Warn when an override for the opposite backend is set.
  • Match packaged library filenames strictly and support compact tags with double-digit major versions.
  • Select binaries in this order:
    1. exact version;
    2. newest older binary from the same major;
    3. oldest newer binary from the same major;
    4. for ROCm only, apply the same older-first policy across majors with a stronger compatibility warning.
  • Keep CUDA fallback restricted to the same major version.

Diagnostics and documentation

  • Report torch.version.rocm and torch.version.hip separately.
  • Document build-time detection, runtime overrides, and compatibility fallback behavior.
  • Add focused CUDA/ROCm tests for exact selection, same- and cross-major fallback, dotted/compact overrides, invalid overrides, and wrong-backend overrides.

Validation

  • pytest -q tests/test_cuda_setup_evaluator.py — 19 passed
  • Ruff and formatting checks pass for modified Python files
  • bash -n .github/scripts/build-rocm.sh
  • ROCm CMake configure succeeds using the wheel-installed SDK and .info/version
  • Local ROCm native build and bitsandbytes GPU sanity check succeed

Compatibility

There is no backend rename in this PR: downstream builds using -DCOMPUTE_BACKEND=hip continue to work. ROCm cross-major fallback is best-effort and emits a warning because compatibility and packaged GPU targets are not guaranteed.

@sstamenk

sstamenk commented Mar 4, 2026

Copy link
Copy Markdown
Contributor Author

Providing a table with various scenarios to better understand the logic. The library is named at build time and looked up by name at runtime. If the names don't match, loading fails.

Runtime: torch.version.rocm available (new PyTorch) → looks for _rocm71 Runtime: only torch.version.hip (older PyTorch) → looks for _rocm64 Runtime: BNB_ROCM_VERSION=71 → looks for _rocm71
Build: .info/version exists → builds _rocm71 ✅ Loads ❌ Fails - rebuild with -DROCM_VERSION=64 or BNB_ROCM_VERSION=71 ✅ Loads
Build: -DROCM_VERSION=71 → builds _rocm71 ✅ Loads ❌ Fails - rebuild with -DROCM_VERSION=64 or BNB_ROCM_VERSION=71 ✅ Loads
Build: hipconfig fallback → builds _rocm64 ❌ Fails - rebuild with -DROCM_VERSION=71 or run with BNB_ROCM_VERSION=64 ✅ Loads (both wrong but agree) ❌ Fails - override points to 71 but lib is 64

When both -DROCM_VERSION and BNB_ROCM_VERSION are listed as fixes, BNB_ROCM_VERSION is preferred as it doesn't require a rebuild.

In most cases we will hit the first scenario, and everything will be fine. Going forward the ROCm version will become even more common. If for some reason, there is a mismatch in version the user can always override them with either -DROCM_VERSION or BNB_ROCM_VERSION.

Going forward the hip version fallback can be entirely removed once a few versions pass and it is no longer needed for compatibility.

@sstamenk
sstamenk force-pushed the fix/rocm-build-rework branch from 91c27f2 to eb72c84 Compare March 4, 2026 20:50
@github-actions

github-actions Bot commented Mar 5, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@matthewdouglas matthewdouglas added this to the v0.50.0 milestone Mar 5, 2026
@sstamenk

sstamenk commented Mar 6, 2026

Copy link
Copy Markdown
Contributor Author

#1889 should address issues seen when using TheRock builds (712 -> 82), this is something I have missed.

@matthewdouglas We might want to hold off a bit on the CMake changes for a later date. The PR changes to the backend naming from hip to rocm people to switch to rocm when building the source and the fallback logic adds a lot of code for not much gain. As long as both runtime look up and build time look up check the HIP version, there shouldn't be an issue in naming. If you want, I can make a separate PR with the unit test changes only or modify this one to just have those changes.

@matthewdouglas

Copy link
Copy Markdown
Member

@sstamenk I agree, let's do any changes (ie rename backend hip -> rocm) on the CMake side separately. I'm going to merge #1889. I'm OK either way if you want to update this PR or just open a new one.

I'm curious, what version of PyTorch adds torch.version.rocm?

@sstamenk

sstamenk commented Mar 9, 2026

Copy link
Copy Markdown
Contributor Author

@matthewdouglas It's available from version 2.10 https://github.com/pytorch/pytorch/blob/release/2.10/tools/generate_torch_version.py#L122

Windows only recently made it possible to do ROCm version lookup during build time (previously only HIP version was available via hipconfig), newer versions of ROCm have the $ROCM_PATH/.info/version file which has the ROCm version so waiting a bit is probably the right move so we can avoid the fallback and just use the ROCm version in the future.

Use ROCm release metadata consistently at build and runtime while keeping packaged-binary fallbacks explicit.
@sstamenk
sstamenk force-pushed the fix/rocm-build-rework branch from eb818ab to efd04b6 Compare August 14, 2026 14:56
@sstamenk
sstamenk marked this pull request as ready for review August 14, 2026 15:04
@sstamenk

Copy link
Copy Markdown
Contributor Author

I've decided to revive this PR after some issues were raised with library detection during testing of one of the upcoming ROCm releases. I've updated the PR description to match the current behavior.

Document the current same-major and ROCm cross-major selection priority more explicitly.
Illustrate older and newer same-major fallback ordering in the loader docstring.
Only pass shared libraries that exist to the packaging copy step.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants